AudioEar: Single-View Ear Reconstruction for Personalized Spatial Audio

Spatial audio, which focuses on immersive 3D sound rendering, is widely applied in the acoustic industry. One of key problems current spatial audio rendering methods lack personalization based different anatomies individuals, essential to produce accurate source positions. In this work, we address problem from an interdisciplinary perspective. The strongly correlated with shape human bodies, particularly ears. To end, propose achieve personalized by reconstructing ears single-view images. First, benchmark ear reconstruction task, introduce AudioEar3D, a high-quality dataset consisting 112 point cloud scans RGB self-supervisedly train model, further collect 2D composed 2,000 images, each one manual annotation occlusion and 55 landmarks, named AudioEar2D. our knowledge, both datasets have largest scale best quality their kinds for public use. Further, AudioEarM, method guided depth estimation network that trained synthetic data, two loss functions tailored data. Lastly, fill gap between vision acoustics community, develop pipeline integrate reconstructed mesh off-the-shelf body simulate Head-Related Transfer Function (HRTF), core rendering. Code data are publicly available https://github.com/seanywang0408/AudioEar.